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Method of Detecting Flaws in the Structure of a Surface 



BACKGROUND OF THE INVENTION. 
1. Field of the Invention. 

10 The invention, in general, relates to a novel method of detecting flaws in 

the surface structure of an object and, more particularly, to a method of detecting 
flaws in the structure of a surface by comparison with a flawless surface stored in 
an artificial neuronal net as a virtual master. Preferably, the invention is applied 
to such tasks as measuring errors which are very small relative to the size of the 

15 test object and where for reasons of economic efficiency the measuring position 
of the test object can only be adjusted with only a low degree of accuracy. Such 
measuring tasks are encountered, for instance, in monitoring the production of 
large surface deep drawn sheet components. 

20 2. The Prior Art. 

A method is known from German patent specification DE 197 53 620.4 C 
in which three-dimensional measuring data of the surface of a test object are 
modified by an artificial neuronal net as a virtual master as if three-dimensional 
25 measuring data were generated of a flawless test object. By comparing the 

original three-dimensional measuring data against the artificially generated three- 
dimensional measuring data flaws, errors and deviations can be detected in the 
surface. 
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It is however, a drawback that for detecting flawed spots it is necessary to 
define three-dimensional data which explicitly characterize the surface structure. 
These three-dimensional data are often calculated on the basis of digital image 
data which also characterize, however implicitly, the surface structure including 
its flawed spots. For the detection of the flawed spots this form of surface 
characteristic is sufficient. 

The complexity of generating three-dimensional measuring data of the 
surface of the test object is greater as a rule than is the complexity of recording 
image data. Often the three-dimensional data are calculated on the basis of 
similar or the same images which in accordance with the invention are utilized 
directly. 

For calculating three-dimensional data of digital images the following 
methods are particularly well suited: A method of the kind described in DE 196 
23 172 C1, a phase shift method of images from two cameras and a the phase 
shift method utilizing a camera and a projector. 

In the calculation of three-dimensional data by one of the mentioned 
methods it is an important prerequisite that the two cameras or the camera and 
the projector in the phase shift method are precisely calibrated. This requires 
additional software. Calibration errors usually result in distortions of the three- 
dimensional data. 

Compared to methods utilizing two cameras, there is a saving of one 
camera. 

OBJECTS OF THE INVENTION. 
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It is an object of the invention to provide a method in which the effect or 
influence of differences in calibration and tolerable variations in the shape of the 
test object is substantially or wholly eliminated, so that with a calibration of low 
complexity even small flaws or errors may be automatically detected by 
5 application of a neuronal net without any explicit calculation of three-dimensional 
data of a test object. In this connection, typical calibration differences and typical 
tolerable form variations are greater than the flaw to be detected. 

SUMMARY OF THE INVENTION. 

10 

In the accomplishment of these and other objects the invention provides 
for a method of detecting or recognizing deviations in the surface structure from 
a predetermined structure by the detection of measurement values and 
subsequent processing by way of an artificial neuronal net including the steps of 

15 projecting a pattern onto the surface and recording images thereof with a matrix 
camera which generates a set of n sequential images while the projected pattern 
is shifted or rotated or changed in its structure by predetermined values, of 
defining from the grey values of a sequence thereof of individual pixels of the n 
images recorded by the matrix camera a number or numbers less than n which 

20 are characteristic of the grey value sequence of a given pixel or the grey value 
sequence of the pixel relative to a grey value sequence or several grey value 
sequences of other pixels, of recalling the neuronal net after the recorded image 
data or matrix of the characteristic numbers of the surface have been input in the 
neuronal net and of utilizing as significant data in respect of deviation values 

25 derived from comparing the image data or the matrix of the characteristic 

numbers of the surface derived from the image data against the recall data of the 
neuronal net. 



By utilization of a neuronal net it is thus possible without significant 
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calibration complexity automatically to detect small errors, flaws or deviations 
without having explicitly to calculate three-dimensional data of the test object or 
surface. Typical calibration differences and typical tolerable variations of form 
have been found to be greater than the errors to be detected. 

5 

DETAILED DESCRIPTION OF THE INVENTION. 

The invention will hereinafter be described in greater detail with reference 
an embodiment: 

10 

The basic problem in seeking to evaluate a test object in respect of 
possible flaws, errors or deviations from a standard object is as follows: 

The objects usually are industrially manufactured ones and their shapes 
15 vary as a result of production technology. Mathematical descriptions of a 
surface, e.g. computer assisted design (CAD) models or a series of flawless 
actual master parts are usually available. The problem does, however, not only 
relate to flawed surface formations; sometimes the problem also relates to the 
position or orientation of a part. Since appropriate data relating to the test object 
20 is not readily available it is, therefore, not possible simply by differentiation 
between the test object and the data of a corresponding master part to extract 
flaws or deviations. The crux of the invention resides in deriving, by calculations 
based on data relating to a test object, data for a virtual master part. In the 
present context, a master part connotes a part without flaws but otherwise in the 
25 same position or orientation, global formation and execution as the test object. 

This is accomplished during the recall phase of a neuronal net. A 
neuronal net deemed particularly suitable for purposes of the invention is an 
associative memory of the kind described in U.S. Application No.: 09/204,485 
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filed on 3 December 1998 by Michaelis et al. and now specifically incorporated 
by reference herein, which receives its task related configuration during a 
training phase. 

5 Recall Phase: 

The algorithm for extracting flaws, errors or deviations is initiated by the 
recording n number of images of the surface or test object to be evaluated by a 
digital camera. Such images implicitly contain three-dimensional data of the 
10 surface. 

To supplement the recording of those images, a projector sequentially n 
number of patterns onto the surface. This is deemed to be an efficient operation 
for purposes of increasing the amount of data. The n number of patterns usually 
1 5 are striped patterns which are shifted by the n th part of the period of the stripes. 
Completely different patterns, such as, for instance, stochastic ones which differ 
in shape or position, may also be utilized. 

As a rule it is efficacious to normalize the brightness values of the 
20 recorded images as lighting conditions may change between different 

measurements. Such changes may occur because of brightness changes by the 
projector or other reflective properties of the surface or test object to be 
measured. One practical way of normalizing the brightness values of the images 
is to use so-called dark images and light images. To this end, the projector 
25 projects a very bright image and a further image with the projector lamp burning 
but the projection path darkened as much as possible as if "black stripes" only 
were being projected. In this manner there would be an upper and a lower 
brightness value for each pixel upon which the grey values of all pixels of the n 
number of recorded images are normalized in a conventional manner, such as, 
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for instance, in relation to the difference between the upper and the lower 
brightness value. Hereinafter, the term "image" will include the images 
normalized in this or a similar manner. 

5 The surface to be evaluated causes the patterns to be spatially deformed. 

Thus, the recorded images implicitly contain three-dimensional data regarding 
the surface. In this connection it is essential that the camera record the image 
from a different direction than the projection path. The flaws or deviations are 
then extracted from the data by the following steps. 

10 

Initially, there is the problem of the great amount of data, n number of 
images are present. Accordingly, n grey values and, hence, a sequence of grey 
values is available for each pixel. This large amount of data might lead to 
difficulties during processing by the associative memory. 

15 

For this reason it is efficacious to reduce the quantity of data. To this end 
the grey value sequence of a pixel is reduced to a characteristic number. This 
may be done in the following ways: 

20 One way is to insure that patterns of n stripes of an almost sinusoidal 

brightness curve are projected vertically of the stripes. While the patterns are of 
identical shape, they are each shifted by the n th part of the period. Known phase 
shift methods may thus be utilized. 

25 The result of specific calculations there will be a phase value for each 

pixel. The phase value relates to the position on one of the sine patterns relative 
to the zero passage of the sine function. The phase value is a characteristic 
number for the entire grey value sequence of the pixel. 
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Another way of calculating a characteristic number is to define the 
similarity of the grey value sequence of the selected pixel relative to the grey 
value sequence of a neighboring pixel. 

5 The characteristic number is, for instance, the cross-correlation coefficient 

between the two grey value sequences. For applying this method to the entire 
image, the pixels to left or to the right of given pixels are utilized. In this manner, 
cross-correlation coefficients are defined for the entire image, except for a 
marginal column or line. These cross-correlation coefficients are modulated by 
10 the three-dimensional shape of the surface. 

The manner of reducing the quantity of data is not restricted to the 
described possibilities. Other processes may also be used. 

15 The data may be processed by the associative memory as follows: 

One of the described matrices of characteristic numbers will serve as 
input data. One phase value for each pixel, one correlation value or another 
suitable characteristic number or the grey values of the n images serve as input 

20 data sets for each associative memory. Hence, n differently configured 

associative memories have to be used. Alternatively, all grey values of the n 
images may serve as input data for one associative memory. This would mean, 
however, that such an associative memory would have to be rather more 
complex than in the previous case, since n relevant data, i.e. grey scale value, 

25 are present for each pixel. 

Each one of the described matrices implicitly describes the three- 
dimensional shape of the surface to be evaluated. The corresponding matrix will 
now be modified by the associative memory in the following manner. A matrix is 
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being generated which could originate with a test object which has no flaws in its 
surface configuration. It is of the same shape and position as the test object and 
is called "virtual master part". 

5 Thereafter, it is only necessary to form the difference between the 

matrices. The flaws will be apparent in the difference; but it is subject to noise. 
Flaws greater than the noise may thus be extracted by simple threshold 
formation. The location of the flaws on the surface or the three-dimensional 
coordinates may then be calculated in a well-known manner. 

10 

The Training Phase: 

This is the phase during which the associative memory is specially 
configured in relationship to the test objects. 

15 

"Training" is conducted on the basis of the implicit three-dimensional data 
of actual flawless master parts. These data have to be of the same kind as 
those of the recall phase, i.,e., they have to be grey values or characteristic 
numbers of the grey value sequences. The training causes the weight factors to 
20 be set or tuned. 

Provided the weight factors have been favorably tuned the associative 
memory will be able on the basis of the data from the test object to generate the 
data of the associated virtual master part. 

25 
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